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Abstract 


The  role  of  schedule  is  fundamental  to  the  acquisition  of  a  particular  system.  This 
topic  is  of  even  more  importance  to  acquisition  in  a  system-of-systems  environment. 
This  report  examines  the  issue  of  schedule  considerations  for  interoperable  acquisi¬ 
tion.  First,  a  Gedanken  red  team  project  is  used  to  explore  concerns  about  schedule  in 
interoperable  acquisition.  Then,  those  concerns  are  examined  in  light  of  current 
requirements  regarding  schedule.  From  that  examination,  several  research  questions 
are  proposed. 
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1  Introduction 


There  is  little  question  of  the  importance  of  a  scheduleto  an  acquisition,  and  much 
has  been  written  about  it;  see,  for  example,  theScheduling  Guide  for  Program  Man- 
agers  [DSMC  01],  Thetermschedu/ecan  convey  different  meanings,  depending  on  the 
context.  One  definition  is 

schedule:  a  series  of  things  to  be  done  in  a  sequence  of  events  within  a  given 
period;  a  timetable  [DAU  05a]. 

The  importance  of  a  schedule  is  that  it  sets  dates  of  major  importance  to  the  organiza¬ 
tion^)  responsible  for  it.  However,  a  schedule  also  creates  expectations  in  other  orga¬ 
nizations. 

This  report  addresses  the  topic  of  schedule  in  the  context  of  interoperable  acquisition. 
We  introduce  the  following  definition: 

interoperable  acquisition:  the  set  of  practices  that  enable  acquisition,  develop¬ 
ment,  and  operational  organizations  to  more  effectively  collaborate  to  field 
interoperable  systems.  This  is  achieved  through  sharing  relevant  information  and 
performing  necessary  activities  that  enable  the  collective  behavior  of  these  orga¬ 
nizations  to  successfully  deliver  systems-of-systems  capabilities. 

An  overview  discussion  of  the  challenges  of  interoperable  acquisition  is  presented  in 
I  interoperable  Acquisition  for  Systems  of  Systems:  TheChallenges.  That  technical  note 
outlines  the  critical  aspects  of  systems  of  systems  and  interoperability  that  affect  the 
acquisition,  deployment,  sustainment,  and  operational  use  of  systems  of  systems 
[Smith  06], 1 

Schedule  concerns  are  of  special  importance  to  interoperable  acquisition  for  several 
reasons,  including: 

•  Several  distributed  organizations  (such  as  program  offices,  decision  agencies,  con¬ 
tractors,  and  users)  often  have  interest  in  a  schedule,  motivated  by  dependencies 
among  them.  These  organizations  may  not  be  known  to  one  another  until  later  in 
the  development,  revealing  the  need  to  address  issues  of  trust  and  culture. 

•  There  is  a  need  to  exchange  information  about  schedule  and  its  dependencies  to 
parties  of  interest.  H  owever,  terminology  used  to  describe  a  schedule  may  be  dif¬ 
ferent  across  different  organizations. 

•  Changes  made  to  a  schedule  by  one  organization  can  have  ramifications  for  other 
organizations  that  have  dependencies  reflected  in  the  schedule. 

Each  of  those  reasons  pushes  schedule  considerations  to  the  forefront  in  any  discus¬ 
sion  of  interoperable  acquisition. 


1 .  In  addition  to  this  technical  note  on  schedule  considerations  and  Interoperable  Acquisition  for  Systems  of  Systems:  The 
Challenges  (CMU/SEI-2006-TN-034),  there  are  two  other  technical  notes  resulting  from  an  independent  research  and  de¬ 
velopment  (IR&D)  project  supported  by  the  Software  Engineering  Institute  (SEI).The  other  technical  notes  describe  pro¬ 
cess  (CMU-SEI-2006-TN-033)  and  risk  management  (CMU/SEI-2006-TN-032)  considerations.  A  fifth  technical  note, 
partially  supported  by  the  IR&D  effort,  will  deal  with  programmatic  interoperability  issues. 
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This  technical  note  is  organized  as  follows: 

•  Section  2  provides  some  background  on  interoperable  acquisition. 

•  Section  3  provides  a  motivating  example  of  a  Gedanken  red  team  looking  at  ques¬ 
tions  of  schedule. 

•  Section  4  looks  at  elements  of  a  schedule  from  the  viewpoint  of  interoperable 
acquisition. 

•  Section  5  provides  a  retrospective  view  and  identifies  some  research  questions. 

•  Section  6  contains  a  brief  summary  of  the  report. 
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2  Background 


2.1  NETWORK-CENTRIC  AND  SYSTEMS  OF  SYSTEMS 

The  terms  network-centric  and  systems  of  systems  have  received  consi  derabl  e  noti  ce  i  n 
the  commercial  and  U  .S.  Department  of  Defense  (DoD)  domains.  These  terms  are 
used  in  contrast  to  monolithic  system— 3  system  designed  for  a  particular  set  of  tasks, 
managed  by  a  single  agency,  and  composed  of  components  that  are  tightly  coupled. 
The  hope  in  the  commercial  and  DoD  domains  is  that  a  network-centric,  system-of- 
systems  approach  will  permit  flexibility  in  dealing  with  changes  in  requirements, 
technology,  or  operational  environment  and  reduce  the  time  needed  to  provide  a  new 
capability. 

I  n  essence,  network-centric  operations  attempt  to  derive  power  from  distributed, 
interacting  entities  based  on  a  significantly  improved  access  to  information.  For 
example,  network-centric  operations  are  expected  to  feature 

•  shared  awareness  through  the  fusion  of  data  from  many  different  types  of  sen¬ 
sors 

I  n  this  context,  the  phrase  common  operational  picture  is  often  encountered. 

•  virtual  collaboration  among  organizations  designed  to  accomplish  a  specific 
purpose 

■  execution  of  activities  by  other,  often  distributed,  organizations  consis¬ 
tent  with  management  intent  [Alberts  99] 

Theterm  network-centric  operations  is  a  postulateof  principles  such  as  shared  aware¬ 
ness.  In  contrast,  theterm  system  of  systems  represents  the  realization  of  those  con¬ 
cepts  in  some  operational  context.  There  are  various  definitions  for  systems  of 
systems,  including  the  foil  owing: 

system  of  systems:  a  set  or  arrangement  of  interdependent  systems  that  are 
related  or  connected  to  provide  a  given  capability  [Levine  03]. 

A  related  term  is  a  family  of  systems.  In  this  technical  note,  we  will  not  distinguish 
between  the  two  terms.  Some  discussion  about  them  appears  in  Requirements  Man¬ 
agement  in  a  System-of-Systems  Context:  A  Workshop  [M  eyers  06a], 

There  are  a  number  of  characteristics  of  systems  of  systems.  One  set  of  characteris¬ 
tics,  provided  by  Maier,  includes  the  following:2 

•  managerial  independence 

The  management  of  each  system  in  a  system  of  systems  is  independent  of  the 
management  of  the  other  systems. 


2.  A  reference  to  Maier’s  work,  as  discussed  here,  can  be  found  at  http://www.infoed.com/Open/PAPERS/systems.htm. 
Maier  no  longer  specifically  includes  emergent  behavior  and  geographic  distribution  as  fundamental  characteristics.  How¬ 
ever,  emergent  behavior  results  from  considerations  of  autonomy  of  managerial  and  operational  independence.  An  over¬ 
simplifying  view,  but  one  to  keep  in  mind,  is  that  a  system  of  systems  is  characterized  by  loose  coupling  rather  than  the 
traditional  tight  coupling  of  a  monolithic  system. 
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•  operational  independence 

Each  system  within  the  system  of  systems  can  function  usefully  in  the  absence  of 
other  systems. 

•  evolutionary  character 

E  ach  system  withi  n  the  system  of  systems  evolves  i  ndependently  from  other  sys¬ 
tems. 

•  emergent  behavior 

A  system  of  systems'  behavi  or  is  a  consequence  of  the  i  nteracti  ons  among  the  i  ndi - 
vidual  systems  that  compose  it;  the  behavior  is  not  embodied  in  any  particular 
system  but  is  a  consequence  of  the  interactions  that  take  place  among  various  sys¬ 
tems.  This  emergent  behavior  often  appears  at  runtime  and  gives  a  system  of  sys¬ 
tems  a  dynamic  character. 

•  geographic  distribution 

The  systems  in  the  system  of  systems  are  not  required  to  be  located  at  the  same 
place. 

The  combination  of  these  characteristics  means  that  the  policies,  practices,  proce¬ 
dures,  and  techniques  used  to  acquire,  develop,  field,  use,  and  sustain  stand-alone 
systems— while  still  vitally  important— must  be  reinterpreted  for  a  system-of-sys- 
tems  context.  I  n  addition,  new  policies,  practices,  procedures,  and  techniques  must  be 
developed  and  integrated  throughout  the  system  acquisition  life  cycle. 

2.2  INTEROPERABLE  ACQUISITION 

The  concept  of  interoperable  acquisition  was  introduced  in  Section  1.  The  purpose  of 
interoperable  acquisition  is  to  more  effectively  allow  organizations  to  share  informa¬ 
tion  and  perform  activities  that  may  affect  their  collective  behavior  in  achieving 
interoperability.  It  is  generally  agreed  that  the  acquisition  process  is  focused  on  a 
particular  acquisition  program,  leading  to  the  well-known  stovepi  pe  approach  to 
acquisition.  Systems  of  systems,  with  network-centric  character,  are  dramatically  dif¬ 
ferent  from  traditional  systems.  I  nteroperable  acquisition  seeks  to  broaden  the  scope, 
role,  and  interaction  of  participants  who  engage  in  the  acquisition  process.  Simply 
stated,  interoperable  acquisition  is  about  achieving  interoperability  in  the  acquisition 
process. 

The  term  interoperability  has  been  used  primarily  with  respect  to  operational  sys¬ 
tems.  I  n  the  literature,  one  finds  many  examples  where  the  term  interoperability  is 
used.  However,  such  usage  is  almost  entirely  restricted  to  the  domain  of  operational 
systems.  Onethemethat  runs  through  the  current  definitions  is  the  ability  of  systems 
to  work  together.  This  notion  has  led  to  an  emphasis  on  a  syntactic  view  of  i nteropera- 
bility  ("bits  on  a  wire"),  although  semantic  considerations  are  quite  relevant  ("what  do 
the  bits  mean?"). 

However,  we  believe  a  more  general  approach  to  interoperability  is  warranted.  T  reat- 
ing  interoperability  only  in  the  context  of  an  operational  system  limits  its  potential 
application  and  benefit.  I  nteroperability  is  about  the  communicating  entities,  the 
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i  nformati  on  they  share,  and  the  operati  ons  that  are  performed  based  on  that  i  nforma- 
tion.  T oward  this  end,  we  offer  a  more  general  definition: 

interoperability:  The  ability  of  a  set  of  communicating  entities  to  (1)  exchange 
specified  information  and  (2)  operate  on  that  information  according  to  a  specified, 
agreed-upon,  operational  semantics. 

Although  it  applies  to  the  context  of  operational  systems,  the  above  definition  is 
intended  to  encompass  a  broader  scope  of  interoperability  in  systems  of  systems.  This 
perspective  is  shown  in  Figure  1,  a  view  that  was  developed  in  earlier  work  ([Levine 
03],  [Meyers  05],  [Meyers  05],  [Smith  05])  and  leads  to  the  consideration  of  program¬ 
matic  interoperability  and  constructive  interoperability,  in  addition  to  opera¬ 
tional  interoperability.  We  suggest  that  acquisition  can  be  addressed  in  terms  of 
functional  domains  in  the  management,  construction,  or  operation  of  a  system. 


Program-1  Program-2 


Figure  1:  System-of-Sy stems  Interoperability  (SOSI)  Model 

Programmatic  interoperability,  constructive  interoperability,  and  operational 
interoperability  represent  interoperability  between  different  domains  that  compose 
an  acquisition.  I  n  particular,  these  domains  are  characterized  by 

•  the  entities  (Program-1,  Program-2,  and  so  forth)  that  need  to  communicate 

•  the  data  they  share 

•  the  operati  ons  that  are  performed 

There  can  also  be  interactions  among  the  various  domains  as  well.  An  example  of  this 
would  be  interaction  between  functions  related  to  program  management  and  system 
construction.  Such  interactions  are  in  the  vertical  dimension  in  Figure  1. 
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A  diagram  such  as  Figure  1  can  be  easily  interpreted  in  an  overly  simplistic  manner. 
We  caution  against  such  oversimplification.  While  it  might  be  a  shorthand  to  inter¬ 
pret  programmatic  interoperability  as  interoperability  among  program  management 
organizations,  such  an  interpretation  is  incorrect.  In  fact,  we  define  programmatic 
interoperability  as 

programmatic  interoperability:  interoperability  among  functions  appropriate  to 
the  management  domain,  independent  of  the  organization  that  performs  those 
functions 

This  definition  suggests  that  programmatic  interoperability  may  involve  the  program 
management  office  as  well  as  others  that  are  engaged  in  the  management  of  a  compo¬ 
nent  or  a  system  of  systems— such  as  users,  contractors,  suppliers,  and  so  on.  Occa¬ 
sionally,  there  is  a  partitioning  of  activities  among  the  participants  in  programmatic 
interoperability;  for  example,  contracting  decisions  are  made  by  a  limited  number  of 
organizations.  Definitions  of  other  types  of  interoperability,  namely  constructive 
interoperability  and  operational  interoperability,  can  be  developed  analogously  to 
that  of  programmatic  interoperability. 

Given  the  preceding  examination,  it  is  relevant  to  discuss  key  influences  that  affect 
the  acquisition  process.  Although  there  may  be  many  such  factors,  our  focus  here  is 
on  those  factors  that  closely  relate  to  considerations  regarding  the  basic  elements  of 
interoperability.  Thefollowing  discussion  is  couched  at  a  higher  level;  we  will  exam¬ 
ine  these  issues  in  more  detail  later  in  this  report. 

2.2.1  Scope  of  Interaction 

I  nter operability  in  the  acquisition  process  is  influenced  by  the  scope  of 
interaction  among  organizations  that  participate  in  the  process.  We  consider 
the  basic  models  shown  in  Figure  2. 


(a)  Within  the 
same 

organization 


(b)  Between  different 
organizations 


O 


O 


(c)  Between  different 
organizations,  possibly 
unknown 


Note:  Different  acquisition  functions  are  denoted  by  the  symbols  O  O  • 


Figure  2:  Models  of  the  Scope  of  Interaction 
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The  different  cases  are  described  as  follows: 

•  Case  (a)  represents  the  interoperability  that  is  within  the  context  of  a  particular 
organization.  I  n  general,  this  is  the  easiest  case  since  it  is  within  the  context  of 
one  organization. 

•  By  contrast,  in  Case  (b)  the  interoperability  is  between  different  organizations.  It 
is  assumed  that  the  set  of  communicating  entities  is  known.  Dealing  with  differ¬ 
ent  organizations  brings  in  consideration  of  organizational  policies  and  practices 
that  might  be  in  conflict. 

•  Case  (c)  is  fundamentally  different  in  that  the  identity  of  the  other  organizations 
may  not  be  known.  This  case  is  analogous  to  an  organization  presenting  informa¬ 
tion  to  others  that  may  need  such  information— independently  of  any  prior  agree¬ 
ment  about  which  organizations  should  be  given  that  information. 

The  three  cases  can  be  seen  as  depicting  two  situations  that  are  quite  different.  The 
first  two  cases  are  characteristic  of  a  bounded  environment,  in  that  the  communicat¬ 
ing  entities  and  their  number  are  known  (and  assumed  to  be  relatively  stable  over 
time).  The  last  case  is  emblematic  of  an  unbounded  environment  in  which  the  identity 
and  number  of  communicating  entities  is  not  known.  That  situation,  an  unbounded 
environment,  is  often  found  in  network-centric  operations  and  systems  of  systems. 

2.2.2  Nature  of  Agreements 

I  nter operability  in  the  acquisition  process  is  influenced  by  the  nature  of 
agreements  among  organizations  that  participate  in  the  process.  Various 
types  of  agreements  can  be  considered  as  part  of  achieving  interoperability: 

•  public  law 

•  contractual  relationships 

•  memoranda  of  understanding 

•  implicit  agreements  (For  example,  if  two  organizations  agree  to  conform  to  some 
standard,  they  have  in  effect  entered  into  an  agreement.) 

I  n  some  sense,  any  type  of  agreement  can  be  regarded  as  an  influence  relation.  The 
nature  of  the  influence  is  often  connected  to  the  expected  behavior  of  the  entities  that 
engage  in  an  agreement.  For  example,  a  contract  specifies  expected  behavior  as 
agreed  to  by  the  contracting  agent  and  the  contractor.  Different  types  of  agreements 
are  related  to  the  character  of  an  organization  entering  into  an  agreement.  The  char¬ 
acter  is  determined  in  part  by  the  environment  in  which  the  agreement  takes  place. 
For  example,  when  a  government  agency  enters  into  a  agreement,  that  agreement  is 
determined,  in  part,  by  the  regulations  that  govern  the  behavior  of  the  government 
agency.  Such  regulations  are  different  when  an  agreement  is  undertaken  by  two  com¬ 
mercial  firms.  Some  discussion  of  thesubject  of  organizations  and  their  agreements  is 
presented  in  System-of-Systems N avi gator:  An  Approach  for  M  anagi  ng  System-of-Sys- 
tems  I  nter  operability  [Brownsword  06], 
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2.2.3  Shared  Information 

Interoperability  in  the  acquisition  process  is  influenced  by  the  information 
that  is  shared.  Fundamental  to  any  discussion  of  interoperability  is  the  information 
that  is  shared.  Two  overarching  considerations  are  (1)  what  information  needs  to  be 
shared  and  (2)  who  decides  that  information.  There  are  interesting  parallels  to  the 
factor  concerned  with  the  scope  of  interaction  between  communicating  entities 
described  in  Section  2.2.1:  what  are  the  impli  cations  for  the  data  that  is  shared  in  an 
unbounded  environment  and  can  the  determination  of  necessary  information  be  made 
at  runtime?  Further,  if  that  determination  can  be  made  at  runtime,  we  have  moved  to 
a  dynamic  environment  that  is  considerably  more  challenging— and  interesting— 
than  a  static  one  in  which  the  scope  of  information  to  be  exchanged  is  bounded  and 
known. 

2.2.4  Operations 

I  nter operability  in  the  acquisition  process  is  influenced  by  the  operations 
that  are  performed  by  the  communicating  entities.  Where  entities  desire  to 
interoperate,  the  operations  performed  are  of  ultimate  concern  because  those  opera¬ 
tions  represent  the  behaviors  of  the  communicating  entities.  It  is  reasonable  to  ask: 

•  What  behavior  is  expected,  or  required,  of  entities  that  participate  in  interopera¬ 
ble  acquisition  in  a  bounded  or  unbounded  environment? 

•  Can  those  behaviors  change  over  time? 

•  I  f  behavi  ors  can  change,  what  are  the  i  mpl  i  cati  ons  of  those  changes? 

2.3  DEALING  WITH  SCHEDULE 

Dealing  with  schedule  is  just  one  aspect  of  interoperable  acquisition  and  there  are 
many  others.  I  n  another  report,  we  considered  the  case  of  interoperable  risk  manage¬ 
ment  defined  as: 

interoperable  risk  management:  the  subset  of  interoperable  acquisition  prac¬ 
tices  that  enable  acquisition,  development,  and  operational  organizations  to  iden¬ 
tify,  share,  and  mitigate  risks  that  are  inherent  to  a  system  of  systems  [Meyers 
06b], 

The  purpose  of  interoperable  risk  management  is  to  more  effectively  allow  organiza¬ 
tions  to  share  information  and  perform  necessary  activities  with  regard  to  risk  man¬ 
agement  that  may  affect  their  collective  behavior. 

The  preceding  definition  could  easily  be  modified  for  the  case  of  interoperable  sched¬ 
ule  management.  The  principles  introduced  earlier,  regarding  communicating  enti¬ 
ties,  information  shared,  and  operations  performed  on  that  information,  may  be 
couched  in  the  context  of  schedule,  namely: 

•  What  are  the  communicating  entities  engaged  in  acquisition  with  regard  to 
schedule? 

•  What  schedule  information  needs  to  be  shared? 

•  What  are  the  operations  (or  behaviors)  related  to  schedule? 
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The  above  questions  are  a  specific  case  of  the  more  general  questions  stated  in  the 
other  report  regarding  interoperable  acquisition.  Here,  however,  thefocus  is  on 
schedule  management. 
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3  A  Motivating  Example 


This  section  will  examine  the  role  of  schedule  in  the  context  of  acquisition  in  a  sys- 
tem-of-systems  environment.3  From  this  examination,  one  can  gain  an  understanding 
of  how  a  schedule  is  viewed  not  only  by  participants  in  some  acquisition  but  also  by 
others  outside  that  acquisition.  Concern  for  a  broader  audience  distinguishes  interop¬ 
erable  acquisition  from  traditional  (system-centric)  acquisition. 

3.1  APPROACH 

To  begin  to  examine  the  role  of  schedule  in  an  acquisition,  we  set  up  a  Gedanken  red 
team4  experiment.  We  assumed  a  red  team  was  being  conducted  that  involved  two 
programs  creating  products  or  services  for  an  integration  project.  This  case  is  the 
more  i nteresti ng  (as  opposed  to  schedule  consideration  for  a  program  acting  alone) 
because  it  addresses  the  integration  of  schedule  information. 

The  information  shown  in  Figure  3  was  first  presented  as  a  slide  to  the  red  team. 


The  arrows  shown  in  Figure  3  represent  the  dependencies  between  the  individual 
programs  and  the  integration  project.  They  therefore  reflect  some  deliverable  that  is 
expected  to  be  provided  to  the  integration  effort. 


3.  We  use  the  phrase  acquisition  in  a  system-of-systems  environments  indicate  that  some  acquisition  entity  is  responsible 
for  producing  a  system  that  will  operate  in  a  system-of-systems  environment.  We  do  not  use  the  phrase  acquisition  of  a 
system  of  systems  because  it  could  be  construed  to  imply  a  single  acquisition  of  a  system  of  systems  treated  as  a  single 
unit. 

4.  The  term  Gedanken  red  team  refers  to  a  thought  experiment  involving  a  red  team.  Originally  used  in  the  sense  of  a 
Gedanken  experiment,  it  refers  to  an  imagined  scenario  that  is  used  to  help  gain  an  understanding  of  the  domain  of  the 
experiment.  The  methodology  is  a  priori  as  opposed  to  empirical. 
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I  n  addition,  the  members  of  the  red  team  were  also  presented  with  a  process  perspec¬ 
tive  of  the  effort.  This  information,  let's  suppose,  was  presented  in  a  second  slide  as 
shown  in  Figure 4. 


Figure  4:  Integration  in  a  Process  Context 


The  following  notes  were  provided  about  this  slide: 

•  Each  program  has  a  development  effort  that  is  modeled  using  an  ETVX  (entry, 
task,  validation,  and  exit  criteria)  model. 

•  Each  program  has  a  process  for  schedule  management  about  the  work  performed 
on  the  program  in  order  to  have  an  up-to-date  view  of  the  status. 

•  The  integration  project  expects  certain  input  from  each  of  the  development  pro¬ 
grams  as  indicated  by  the  arrows  in  the  diagram.  The  integration  project  also  has 
a  process  that  performs  the  function  of  schedule  management. 

1 1  was  al  so  stated  that  there  was  some  concern  about  meeti  ng  the  schedul  e  event  for 
the  integration  effort  shown  in  Figure  3.  I  n  particular,  there  was  concern  voiced  that 
the  input  from  Programs  1  and  2  would  not  be  ready  for  the  integration  effort  to  pro¬ 
ceed. 
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Foil  owing  the  presentation  of  the  preceding  information,  the  members  of  the  red  team 
were  invited  to  express  their  concerns.  That  discussion,  largely  couched  in  the  form  of 
questions,  is  presented  in  thefollowing  section. 

3.2  RED  TEAM  QUESTIONS 

A  number  of  questions  were  asked  regarding  the  information  shown  in  Figures  3  and 
4.  The  various  questions  can  be  grouped  in  thefollowing  categories: 

•  basic  information 

•  organizations  and  dependencies 

•  shared  information 

•  approval 

•  risks 

•  dealing  with  change 

The  questions  were  posed  to  a  program  manager  (PM ),  the  integrator  (I  ntegrator),  or 
all  organizations  (All). 

3.2.1  Basic  Information 

A  number  of  questions  arose  simply  regarding  the  interpretation  of  the  material 
shown  in  Figure 3,  including 

•  (All)  What  is  the  meaning  of  the  triangles  shown  in  this  figure?  Are  they  formal 
milestones,  reviews,  or  what? 

•  (All)  Froma  higher  level  perspective,  what  drove  the  choice  of  dates  for  the  sched¬ 
ule  events?  Flow  did  these  choices  fit  the  overall  contracting  strategy? 

•  (PMs)  Flow  did  you  pick  the  date  for  the  schedule  events?  What  are  the  chances 
you'll  meet  those  dates?  Do  you  have  metrics  or  past  performance  information  to 
support  the  confidence  in  your  estimates? 

•  (I  ntegrator)  Flow  did  you  pick  the  date  for  when  PMs  had  to  provide  you  with 
things  for  the  integration?  What  was  the  role  of  the  PMs  in  deciding  this?  What 
confi  dence  do  you  have  that  the  P  M  s  wi  1 1  meet  thei  r  dates? 

•  (All)  Please  describe  the  process  called  schedule  management. 

•  (All)  What  is  the  critical  path? 

3.2.2  Organizations  and  Dependencies 

A  number  of  organizations  participate  in  the  integration  effort.  It  is  natural,  there¬ 
fore,  to  seek  to  understand  their  roles  and  interdependencies.  Thefollowing  questions 
were  asked: 

•  (All)  What  organizations  can  influence  your  schedule  or  your  ability  to  meet  any 
dates? 

•  (All)  Describe  the  nature  of  any  dependencies  with  other  programs  or  organiza¬ 
tions  that  can  affect  a  schedule  event.  Flow  are  these  dependencies  managed  (e.g., 
on  a  one-to-one  or  a  collective  basis)? 
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•  (All)  What  is  the  nature  of  the  ongoing  interactions  between  the  PMs  and  the 
integrator  with  respect  to  schedule?  Is  there  a  description  of  this  process?  Are 
there  practices?  MOUs?5  How  formal  are  they? 

•  (All)  Who  makes  surethe  process  is  performed  accordingto  its  intended  plan? 
Where's  the  belly-button? 

•  (All)  Who  is  the  arbitrator  for  decisions  if  there  are  disagreements;  how  are  dis¬ 
agreements  resolved?  Is  there  an  overarching  authority? 

•  (All)  Are  there  dependencies  on  external  entities  (a  legacy  system,  an  organiza¬ 
tion,  or  a  commercial  off-the-shelf  [COTS]  product)  that  cannot  be  changed?  H  ow 
is  a  case  such  as  this  handled? 

3.2.3  Shared  Information 

Because  the  context  of  this  exercise  deals  with  the  integration  of  work  products  (or 

services)  provided  by  different  PMs  to  an  integrator,  it  is  natural  that  there  would  be 

questions  about  the  information  shared.  Thefollowing  questions  were  asked: 

•  (PMs)  What  is  it  that  you  are  expected  to  provide  to  the  integrator?  Is  there  a 
written  agreement  regarding  this  information?  Who  signed  off  on  it?  When  was  it 
signed?  How  often  has  it  been  modified  and  why? 

•  (I  ntegrator)  Do  you  routinely  monitor  the  progress  of  the  PMs  for  their  ability  not 
onlyto  meet  the  integration  date  but  also  to  provide  quality  products6  to  you? 
How  would  you  know  if  this  were  not  the  case? 

•  (PMs)  What  are  the  exit  criteria  for  products  of  relevance?  Who  developed  them? 
Were  they  approved  by  the  integrator? 

•  (I  ntegrator)  Is  there  a  way  you  can  verify  the  exit  criteria  are  satisfied  for  the 
products  provided  to  you?  Are  the  exit  criteria  from  the  PMs  the  same  as  your 
entrance  criteria  to  the  integration  effort?  If  not,  what  differences  are  there? 

•  (All)  How  harmonious  are  the  exit  criteria  among  the  participants?  Here  are  two 
examples: 

Are  the  exit  criteria  for  the  PMs  weaker  than  the  entrance  criteria  defined  by 
the  integrator? 

Could  the  entrance  criteria  defined  by  the  i ntegrator  be  stronger  than  the 
exit  criteria  defined  by  the  PMs? 

Who  monitors  situations  I  i  ke  these  for  potenti  al  mismatches?  Whi  I  e  the  two  cases 
appear  to  be  similar  (and  will  have  similar  consequences),  their  rationale  is  differ¬ 
ent.  How  are  conflicts  identified  and  resolved?  And  who  pays  to  make  the  prod¬ 
ucts  conform  to  the  entrance  and  exit  criteria  assessments  when  there  is  a 
disagreement? 

•  (PMs)  If  you  want  to  change  an  exit  criterion,  is  the  change  agreed  toby  theinte- 
grator?  Who  pays? 


5.  An  MOU  is  a  Memorandum  of  Understanding.  For  a  listing  of  acronyms  and  initialisms  used  in  this  technical  note,  see  the 
Acronyms  and  Initialisms  section. 

6.  Flere  and  hereafter,  we  will  use  the  term  product  to  mean  either  a  product  or  a  service. 
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•  (Integrator)  If  you  want  to  change  an  entrance  criterion,  is  the  change  agreed  to 
by  an  affected  PM  ?  Who  pays? 

•  (All)  Is  there  any  proprietary  information?  How  is  it  handled? 

•  (All)  Are  there  intermediate  points  where  you  share  information  to  get  an  idea  on 
progress? 

•  (PMs)  Are  there  any  dependencies  between  the  products  you  are  creating,  inde¬ 
pendent  of  the  integrator?  How  are  conflicts  identified  and  managed? 

3.2.4  Approval 

Schedule  events  can  be  of  various  types,  from  formal  reviews  with  other  organizations 

to  events  that  are  localized  to  a  project.  There  may  also  be  requirements  related  to 

the  approval  of  a  schedule  event.  Thefollowing  questions  arose: 

•  (All)  Does  the  schedule  event  require  approval?  If  so,  why  is  this  not  shown  as  a 
mi  I  estone  on  the  si  i  de? 

•  (Al  I )  Who  approves  the  schedul  e  events? 

•  (All)  What  is  the  current  status  of  the  schedule  event(s)  with  regard  to  it  being 
approved? 

•  (All)  What  are  the  consequences  to  the  schedule  event  if  is  not  approved  when 
expected?  I  s  there  another  round  of  approval?  What  are  the  implications  of 
another  round  of  approval  to  the  overall  process? 

•  (All)  What  are  the  criteria  associated  with  the  approval  process?  Who  decides 
them? 

•  (All)  Are  there  artifacts  that  must  be  provided  as  part  of  the  approval  process? 
What  are  they?  Are  these  artifacts  shared  with  others?  I  f  so,  how  are  they  used? 

•  (All)  Are  there  dependencies  among  the  approval  processes  and  if  so,  how  are  they 
managed  (e.g.,  is  it  possiblethat  the  schedule  event  for  the  integrator  could  be 
approved  prior  to  the  approval  of  the  schedule  events  for  the  PMs)?  Who  man¬ 
ages  the  dependency  of  approvals? 

•  (All)  Can  the  approval  for  a  previously  approved  schedule  event  be  withdrawn? 
By  whom?  What  happens?  H  as  this  ever  happened? 

3.2.5  Risks 

It  is  accepted  that  programs  should  manage  risk.  Thus,  it  was  not  surprising  that 

questions  were  generated  around  this  topic,  including 

•  (PMs)  What  are  the  risks  to  your  meeting  the  schedule  event?  What  is  the  basis  of 
risk  and  how  was  it  determined?  Are  the  risks  shared,  and  managed,  among  all 

P  M  s  and  the  i  ntegrator? 

•  (I  ntegrator)  What  are  your  risks,  apart  from  those  directly  related  to  a  PM,  to 
meet  the  integration  schedule  event?  How  were  they  determined? 

•  (All)  How  much  sharing  of  risk  management  information  is  performed?  What  is 
the  process?  Is  it  routinely  practiced?  How  are  decisions  made  about  risk  mitiga¬ 
tion  planning;  in  particular,  who  pays?  What  is  the  level  of  authority  of  parti ci - 
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pants?  Do  they  have  the  ability  to  authorize  change?  What  is  the  scope  of  their 
authority  and  knowledge  to  make  decisions? 

•  (I  ntegrator)  What  happens  if  the  PM  s  do  not  meet  their  schedule?  How  and  when 
will  you  know?  What  is  your  fallback  position?  Do  you  have  any  contingency 
plans? 

•  (All)  What  are  the  risks  to  your  schedule  that  are  outside  the  scope  of  your  control 
(e.g.,  COTS)?  How  do  you  deal  with  these  risks? 

3.2.6  Dealing  with  Change 

Dealing  with  change  is  normal  for  acquisition  programs.  The  ability  to  deal  with  and 
manage  change  becomes  more  important  when  multiple  acquisition  programs  are 
involved.  It  was  natural  that  the  red  team  would  probe  this  point;  the  questions  asked 
included 

•  (PMs)  What  istheimpact  on  you  if  the  integrator  has  to  move  the  schedule  event 
to  the  I  eft?  To  the  ri  ght? 

•  (PMs)  How  do  you  make  the  integrator  aware  of  changes  to  your  schedule  events? 
What  is  the  severity  of  change?  How  do  you  assess  impact  of  change  to  schedule? 
How  does  the  integrator  participate  in  this  process? 

•  (All)  How  do  you  find  out  about  potential  schedule  changes  from  other  organiza¬ 
tions,  like  contractors  or  other  programs? 

•  (Integrator)  How  do  you  communicate  the  need  for  possible  schedule  changes  to 
the  integration  schedule  event  to  the  PMs? 

•  (All)  What  is  the  time-dependence  of  the  entrance  and  exit  criteria?  Do  those  cri¬ 
teria  change  often?  Why?  We  are  interested  in  the  volatility  of  these  criteria. 

•  (All)  Isthere  slack  built  into  your  schedule?  How  is  it  determined?  Is  it  visibleto 
others? 

•  (All)  Do  you  foreseethe  need  to  make  any  contractual  modifications?  If  so,  how 
will  cost  be  managed?  What  happens  to  the  schedule?  Has  this  been  done  in  the 
past?  What  happened? 

3.3  SUMMARY 

Approaching  the  question  of  schedule  by  considering  a  Gedanken  red  team  has  proven 
to  be  interesting.  It  is  worth  stepping  back  and  noting  some  observations  about  the 
results. 

•  The  questions  cover  many  topics,  typical  of  a  red  team.  We  believe  this  reflects 
two  things.  First,  schedule  events  areafocal  point  for  entreeintoa  discussion  of 
many  related  topics.  Second,  the  inherent  importance  of  schedule  events  makes 
them  prime  candidates  for  discussion. 

•  Schedule  has  interest  beyond  some  milestone.  There  is  a  lot  of  information 
reflected  by  a  schedule,  and  it  didn't  take  long  to  uncover  questions  demonstrat¬ 
ing  a  much  larger  scope  than  simply  a  milestone. 
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•  There  is  a  gradation  of  visibility  about  information  derived  from  a  discussion  of 
schedule.  Some  information  might  be  public  and  can  be  shared  with  others;  other 
information  might  bekept  private  with  little  willingness  to  share(e.g.,  duetoloss 
of  a  possible  competitive  advantage).  The  issue  of  the  degree  of  sharing  appears 
often  in  the  preceding  discussion  and  is  no  doubt  a  general  consideration  for  any 
system  of  systems. 

•  The  questions  seem  to  apply  to  an  acquisition  of  a  single  system  as  well  as  acqui¬ 
sition  in  the  context  of  a  system  of  systems.  It  is  necessary,  however,  to  consider 
dependencies,  due  to  the  presence  of  multi  plepartici  pants  in  an  acquisition  in  the 
context  of  a  system  of  systems.  There  may  be  differences  in  management  of  con¬ 
trol  and  authority  and  processes.  Significant  differences  in  these  aspects  might 
indicate  areas  of  concern.  When  multiple  organizations  are  present,  a  holistic  pic¬ 
ture  must  emerge— not  the  view  of  a  single  program  or  organization. 

Considerations  such  as  the  preceding  ones  warrant  a  more  detailed  look  at  the  ele¬ 
ments  that  are  associated  with  a  schedule.  This  material  will  be  discussed  in  thefol- 
I owing  section. 
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4  Schedule 


I  n  this  section,  we  examine  the  subject  of  schedule  from  the  perspective  of  interopera¬ 
ble  acquisition;  in  it,  we  assume  that  the  reader  has  some  familiarity  with  DoD  acqui¬ 
sition.7  We  focus  on  milestones,  including  certain  requirements  imposed  by  the  legal 
and  regulatory  environment,  and  consider  interoperability  aspects  related  to  a  sched¬ 
ule.  While  it  is  oriented  toward  schedule  events  more  often  considered  in  the  context 
of  an  acquisition,  our  discussion  in  this  section  can  also  apply  to  the  development  and 
operational  aspects  of  an  acquisition. 

4.1  BACKGROUND 

The  term  schedule  can  have  various  interpretations,  depending  on  the  context  in 
which  it  is  used.  I  n  the  simplest  case,  a  schedule  is  a  sequence  of  temporal  events  or 
activities.  For  example,  in  the  DoD  a  schedule  is  associated  with  an  acquisition  life 
cycle.  The  life  cycle  is  divided  into  acquisition  phases,  and  there  may  be  milestones 
associated  with  each  phase.  A  simple  rendition  of  an  acquisition  life  cycle  is  shown  in 
Figure  5.  The  life  cycle  shown  is  sequential,  but  other  choices  are  possible,  such  as  an 
evol  utionary  acquisition. 
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A,  B,  and  C  denote  milestones 

CD,  DRR,  and  FRP  denote  decision  reviews  for  Concept  Decision,  Design  Readiness  Review, 
and  Full  Rate  Production,  respectively. 


Figure  5:  Typical  Phases  in  an  Acquisition  Life  Cycle 


To  deal  with  the  specifics  of  a  schedule,  we  represent  an  abstraction  of  the  concept  of 
a  schedule  in  Figure  6.  The  representation  shown  includes  information  typically  pre¬ 
sented  in  a  schedule  traditionally  associated  with  an  acquisition  program. 


7.  The  Introduction  to  Defense  Acquisition  Management  from  the  Defense  Acquisition  University  provides  background  on 
DoD  acquisition  [DAU  05b]. 
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Figure  6:  Abstraction  of  a  Schedule 

The  information  shown  in  Figure  6  suggests  that  a  schedule  can  also  be  viewed  as  a 
collection  of  phases  and  that  events  are  associated  with  a  phase.  An  event  can  be  of 
different  types,  including  a  decision  point  and  others  indicated  in  thefigure.  A 
group  of  decision  points  is  subdivided  further  into  milestone  decisions  and  mile¬ 
stone  reviews.  Among  the  group  of  milestone  decisions  are  those  denoted  A,  B,  and 
C;  mi  lest  one  reviews  include  Concept  Decision  (CD),  Design  Readiness  Review  (DRR), 
and  Full  Rate  Production  (FRP).  Each  event,  such  as  Milestone  A,  has  a  name,  a 
meaning  (semantic  content),  and  some  temporal  representation  (most  often  shown  as 
a  triangle  on  a  Gantt  chart).  Notice  that  the  group  of  events  shown  in  Figure  6  con¬ 
tains  other  reviews  of  various  types  as  well  as  milestone  reviews.  The  simple  taxon¬ 
omy  shown  in  Figure  6  applies  to  the  schedule  shown  in  Figure  5. 

4.2  EVENTS 

A  major  element  of  a  schedule  is  an  event,  and  several  events  are  listed  in  Figure  6, 
including  decision  points.  I  n  thefollowing  sections,  we  consider  characteristics  and 
supplementary  information  that  could  be  associated  with  decision  points. 

4.2.1  Milestone  Decisions 

Consider  first  the  milestone  decision  point,  which  authorizes  entry  of  an  acquisition 
program  into  some  acquisition  phase,  such  as  program  initiation.  The  source  of  infor¬ 
mation  about  a  milestone  decision  can  be  statutes8  or  agency  regulations,  such  as  the 
The  Defense  Acquisition  System  (DoD  Directive  5000.1)  [DoD  05a]  and  the  Federal 


8.  Most  often,  a  statute  (e.g.,  Title  1 0)  specifies  what  must  be  done,  not  when  it  must  be  done.  In  some  cases,  (e.g.,  spec¬ 
ification  of  Low  Rate  Initial  Production  [LRIP]  quantities  as  per  10  USC  §  2400),  there  is  direct  reference  to  a  temporal 
event.  If  a  statute  does  not  define  when  a  particular  item  must  be  provided,  the  date  is  determined  through  regulations 
set  forth  by  the  acquiring  agency  [USC  04], 
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Acquisition  Regulations  (FARs).  The  rationalefor  including  statutory  information  in 
this  report  is  that  we  are  i nterested  in  the  effect  of  statutes,  in  particular  Title  10,  on 
acquisition  in  a  system-of-systems  environment.9  I  n  general,  to  the  extent  that  stat¬ 
utes  refer  to  particular  temporal  events,  they  affect  an  acquisition,  especially  an 
acquisition  in  a  system-of-systems  environment. 

Table  1  indicates  the  required  information,  its  source,  and  the  milestone  decision 
(designated  as  MS  A,  MS  B,  and  MS  C)  to  which  it  is  applicable.  The  information  in 
the  table  was  taken  from  Operation  of  the  Defense  Acquisition  System  (DoD  I  nstruc- 
tion  5000.2)  [DoD  04].10 

Table  1:  Supplementary  Information  for  Milestone  Decisions 


Required  Information 

Source 

MSA 

MS  B 

MS  C 

Acquisition  Program  Baseline 

10  use  §  2435 

✓ 

S 

Benefit  Analysis  and  Determination 

15  USC  644(e) 

✓ 

S 

Clinger-Cohen  Act  Compliance 

40  USC  Subtitle  III 

Pub.  L.  107-248,  §  8088 

✓ 

✓ 

S 

Certification  of  compliance  with  the  Financial 
Management  Enterprise  Architecture 

Pub.  L.  107-248  §  8088 

✓ 

✓ 

S 

Competition  Analysis 

10  USC  §  2469 

✓ 

S 

Consideration  of  Technology  Issues 

10  USC  §  2364 

✓ 

✓ 

V 

Cooperative  Opportunities 

10  USC  §  2350a 

✓ 

Core  Logistics  Analysis/Source  of 

Repair  Analysis 

10  USC  §  2460 

10  USC  §  2464 

10  USC  §  2466 

✓ 

s 

1  ndependent  Cost  Estimates 

10  USC  §  2434 

✓ 

V 

1  ndustrial  Capabilities 

10  USC  §  2440 

✓ 

Live  Fire  Waiver  and  Alternate  Live  Fire 
Testing  and  Evaluation  (LFT&E)  plan 

10  USC  §  2366 

✓ 

LRI  P  quantities 

10  USC  §  2400 

✓ 

Market  Research 

10  USC  §  2377 

15  USC  §  2644(e) 

✓ 

✓ 

Program  Deviation  Report 

10  USC  §  2435 

Immediately  upon  a  program 
deviation 

Programmatic  Environment  Safety  and 
Occupational  H ealth  Evaluation 

40  USC  §  4321 

✓ 

V 

Registration  of  mission-critical  and  mission- 
essential  information  systems 

Pub.  L.  107-248 

Pub.  L.  106-398  §  811 

✓ 

V 

Selected  Acquisition  Report  (SAR) 

10  USC  §  2432 

✓ 

V 

Spectrum  Certification  Compliance 

47  USC  §  305 

47  USC  §  901-904 

✓ 

U  nit  Cost  Report 

10  USC  §  2433 

Quarterly 

Technology  Development  Strategy 

Pub.  L.  107-314  §  803 

✓ 

✓ 

The  information  shown  in  Table  1  lists  only  the  statutes  that  apply  to  a  particular 
milestone.  Beyond  statutory  considerations,  the  DoD  has  developed  regulatory 
requirements.  The  Operation  of  the  Defense  Acquisition  System  (DoD  I  nstruction 


9.  Results  of  an  analysis  of  Title  1 0’s  effect  on  acquisition  in  a  system-of-systems  context  will  be  reported  in  the  future. 

10.  Some  details  associated  with  the  data  taken  from  Operation  of  the  Defense  Acquisition  System  are  not  reported  in  Table 
1 .  For  instance,  requirements  might  vary  for  different  types  of  acquisition  categories. 
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5000.2)  [DoD  04]  specifies  that  exit  criteria  be  provided  for  milestones  A,  B,  and  C,  for 
instance;  other  DoD  regulations  require  an  I  nitial  Capabilities  Document  (I  CD)  and 
Capability  Development  Document  (CDD). 

4.2.2  Milestone  Reviews 

N  ext,  consider  the  case  of  milestone  reviews.  The  purpose  of  such  reviews  is  to  assess 
progress  in  a  program  and  authorize  further  activity.  The  type  of  information 
required  for  a  milestone  review  is  shown  in  Table  2. 


Table  2:  Supplementary  Information  for  Milestone  Reviews 


Required  Information 

Source 

CR 

DRR 

FRP 

Acquisition  Program  Baseline 

10  use  §  2434 

S 

Beyond  LRIP  Report 

10  USC  §  2399 

V 

Certification  of  compliance  with  the 
Financial  Management  Enterprise  Archi¬ 
tecture 

Pub.  L.  107-248  §  8088 

s 

Clinger-Cohen  Act  Compliance 

40  USC  Subtitle  III 

Pub.  L.  107-248  §  8088 

1  ndependent  Cost  Estimate  and  Man¬ 
power  Estimate 

10  U  SC  §  2434 

V 

LFT&E  report 

10  USC  §  2366 

s 

Post-Deployment  Performance  Review 

5  USC  §  306, 

40  USC  §  1131 

s 

Programmatic  Environment  Safety  and 
Occupational  H ealth  Evaluation 

41  USC  §  4321 

s 

Registration  of  mission-critical  and  mis¬ 
sion-essential  information  systems 

Pub.  L.  107-248  §  8088 

Pub.  L.  106-398  §  811 

s 

Selected  Acquisition  Report  (SAR) 

10  USC  §  2432 

V 

The  information  shown  in  Table  2  is  required  by  statute,  in  particular  by  Title  10. 

Th  eOperation  of  the  Defense  Acquisition  System  (DoD  I  nstruction  5000.2)  [DoD  04] 
does  not  include  references  for  statutes  regarding  the  Concept  Review  (CR)  and  DRR 
events.  However,  DoD  regulations  may  place  requirements  on  these  reviews.  For 
example,  the  Operation  of  the  Defense  Acquisition  System  (DoD  Instruction  5000.2) 
requires  an  Analysis  of  Alternatives  (AoA)  plan  as  part  of  the  CD  review. 

Other  requirements  specified  in  Operation  of  the  Defense  Acqui  si  ti  on  System  (DoD 
I  nstruction  5000.2)  [DoD  04]  and  other  regulations  apply  to  milestones  A,  B,  and  C, 
such  as 

•  Acquisition  Strategy 

•  Command,  Control,  Communications,  Computers,  and  I  ntelligence  (C4I )  Support- 
ability  Certification 

•  Economic  Analysis 

•  Component  Cost  Analysis 

•  Cost  Analysis  Cost  Description 

•  Test  and  Evaluation  Master  Plan 

•  Operational  Test  Agency  Report  of  Operational  Test  and  Evaluation  Results 
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4.2.3  Other  Events 

Thus  far,  we  have  considered  information  required  by  statute,  in  particular  by  Title 
10.  There  are  other  schedule  events  called  out  in  regulatory  documents  (and  shown  in 
Figure  6  on  page  20),  including  the  foil  owing: 

•  DAB  review 

•  executive  reviews 

•  J  ROC  reviews 

•  ITAB  reviews 

I  n  addition,  other  events  conducted  for  a  particular  program  are  often  shown  in  the 
detai  I  s  of  the  schedul  e  of  the  program.  Some  possi  bl  e  events  of  thi  s  type  are 

•  risk  management  reviews 

•  cost  revi  ews  i  nd  udi  ng  i  ndependent  cost  esti  mates 

•  evaluation  of  a  COTS  product 

•  product  status  reviews,  including  obsolete  or  unattainable  products 

The  preceding  events  might  become  part  of  some  other  event  that  is  required  to  be 
reported  and  reviewed.  For  example,  a  program  might  perform  internal  cost  reviews; 
the  results  of  these  reviews  might  be  included  later  in  a  SAR. 

4.2.4  Summary 

A  myriad  of  information  is  required  to  be  reported  with  a  schedule.  Some  of  this  infor¬ 
mation  is  based  on  statute;  some,  on  regulations.  The  amount  of  information  required 
demonstrates  the  importance  of  a  scheduletoa  particular  program.  Flence,  it  is  natu¬ 
ral  to  assume  the  same  importance  is  present  when  one  considers  multiple  programs 
participating  in  an  acquisition  (i.e.,  in  a  system-of-systems  environment). 

4.3  INTEROPERABILITY  CONSIDERATIONS 

It  is  relevant  now  to  look  at  the  questions  developed  by  the Gedanken  red  team  (see 
Section  3)  in  terms  of  interoperable  acquisition.  In  thefollowing  sections,  we  focus  on 
the  three  main  aspects  of  interoperability:  (1)  communicating  entities,  (2)  information 
shared,  and  (3)  behaviors. 

4.3.1  Communicating  Entities 

A  natural  approach  to  identifying  entities  that  seek  to  communicate  schedule  infor¬ 
mation  can  be  determined  from  our  discussion  of  events.  Restricting  our  attention  to 
the  case  of  milestone  reviews,  we  suggest  that  the  entities  would  include  the  foil  ow¬ 
ing: 

•  Program  Management  Office  (PM  O) 

This  PMO  has  acquisition  authority  and  responsibility  for  the  schedule  elements 
(in  this  case,  events).  Note  that  a  PMO  is  almost  always  tied  to  a  particular  sys¬ 
tem. 

•  PM  Os  with  which  the  system  is  expected  to  i nteroperate 

F  or  exampl  e,  there  may  be  a  need  to  perform  i  ntegrati  on  testi  ng  of  the  systems. 
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•  Program  Executive  Office  (PE  O) 

A  PEO  provides  some  form  of  oversight  to  programs  and  often  has  interest  in  the 
progress  of  the  program. 

•  Milestone  Decision  Authority  (M DA) 

The  M DA  must  approve  milestone  decision  points. 

•  J  ROC 

TheJ  ROC  represents  the  operational  community  and  is  interested  in  the  progress 
of  a  program. 

Possibly,  other  organizations  can  be  included,  too.  Thus,  we  extend  the  preceding  list 
to  include  these  entities: 

•  contractor  management 

•  user  representatives 

•  sustainment  agency 

•  training  agency 

•  test  and  evaluation 

•  deployment  agency 

•  oversight  agency11 

4.3.2  Information  Shared 

One  key  aspect  of  interoperability  is  the  information  that  must  be  shared  among  enti¬ 
ties.  Figures  5  and  6  (on  pages  19  and  20)  illustrate  the  concepts  that  need  to  be  iden¬ 
tified  with  regard  to  a  schedule.  However,  what  information  is  associated  with  each  of 
these  items?  For  a  schedule  event,  we  suggest  that  the  foil  owing  are  relevant: 

•  the  name  of  the  particular  schedule  event 

The  name  must  be  unique  across  all  instances  of  relevant  schedule  events. 

■  the  semantics  (meaning)  of  the  particular  event 

I  n  the  case  of  Milestone  A,  one  choice  for  the  semantics  is  "the  point  at  which  a 
recommendation  is  madeand  approval  sought  regarding  starting  or  continuing  an 
acquisition  program,  i.e.,  proceeding  to  the  next  phase.  Milestone  A  approves 
entry  into  the  Technology  Development  (TD)  phase"  [DAU  05a], 

•  contact 

There  may  be  a  need  to  obtain  information  about  a  particular  schedule  event,  par¬ 
ticularly  in  a  system-of-systems  context.  The  means  of  contact  may  be  by  tele¬ 
phone,  email,  or  Web  page.  (Note  that  the  contact  need  not  be  a  person.) 

■  date  and  milestone  probability  distribution  function 

Traditionally,  a  milestone  has  been  represented  as  a  point  in  time.  However,  as 
we  all  know,  that  is  a  bit  unrealistic.  I  n  the  general  case,  the  "date"  for  a  mile- 


1 1 .  The  role  of  an  oversight  agency  always  causes  contention.  For  example,  in  a  recent  review  of  a  NOAA  satellite  program, 
it  was  stated  that  “inadequate  management  oversight,  in  effect,  postponed  critical  evaluations  and  decisions  needed  to 
replan  the  program's  faltering  elements  and  contain  cost  and  schedule  overruns....  Time  and  money  were  thus  wasted” 
(Federal  Computer  Week,  May  15,  2006,  available  at  http://www.fcw.com/article94525-05-15-06-Web). 
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stone  can  be  regarded  as  a  probability  distribution  function  (pdf).  Several  example 
representations  are  shown  in  Figure  7. 
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time 
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Figure  7:  Some  Different  Representations  of  a  Milestone 
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Case  (a)  shows  the  traditional  approach— a  milestone  represented  as  a  single 
point  in  time.  Case(b)  represents  a  milestone  as  a  Gaussian  (normal)  distribu¬ 
tion  that  has  some  mean  and  standard  deviation.  Another  choice  is  shown  in 
Case  (c),  which  is  a  skewed  distribution;  the  typical  interpretation  for  a 
skewed  distribution  is  that  the  milestone  cannot  come  before  some  date  and  is 
most  likely  to  occur  after  a  specified  date.  Finally,  Case  (d)  shows  an  equal 
probability  over  some  interval.  In  any  case,  inclusion  of  a  pdf  reflects  the  pos¬ 
sible  risk  to  achieving  a  particular  date. 

A  date  associated  with  a  milestone  should  not  be  considered  as  a  single  point 
in  time.  I  nstead,  we  must  recognize  the  inherent  possibility  of  variation  when¬ 
ever  a  date  is  specified.  F urthermore,  we  need  to  be  interested  in  not  only  one 
pdf,  but  also  in  joint  distributions  that  must  account  for  dependencies  among 
the  participants.  For  example,  if  a  program  has  a  schedule  indicating  when  it 
is  supposed  to  deliver  something  to  an  integration  effort  (which  also  has  a 
schedule),  it  is  the  joint  distribution  that  is  more  reflective  of  the  collaboration 
of  the  two  organizations. 

■  the  status  of  the  schedule  event 

The  event  status  indication  might  be  limited  to  values  such  as  Pending, 

I  n_Pr ogress,  and  Complete. 

•  approval  information 

Most  milestones  have  some  information  associated  with  event  approval.  For 
example,  a  milestone  decision  review  requires  approval  by  the  M  DA.  As  a  mini¬ 
mum,  we  would  suggest  that  thefollowing  information  is  relevant  regarding 
approval: 

approval  status 

An  indication  of  the  state  associated  with  the  approval  process,  such  as  the 
values  U napproved,  ln_Progress,  and  Approved. 

approving  office 

This  office  has  authority  to  approve  the  schedule  event. 
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approving  individual 

This  individual  has  authority  to  approve  the  event, 
approval  date12 

This  date  is  when  approval  was  granted. 

The  tuple  of  name,  meaning,  contact,  milestone  pdf,  status,  and  approval  information 
applies  to  all  decision  reviews  and  to  other  events  (although  not  all  other  events  apply 
to  the  concept  of  a  phase).  We  recognize  that  there  are  two  types  of  decision  points, 
milestone  decisions  and  milestone  reviews,  and  find  that  different  supplementary 
data  is  associated  with  each  type  (due  to  statutory  or  regulatory  considerations).  The 
differences  were  noted  in  the  discussion  of  Tables  1  and  2  on  pages  21  and  22. 

Each  item  associated  with  a  schedule  has  an  associated  syntax  and  semantics.  The 
broader  the  scope  of  the  participants  (e.g.,  from  a  bounded  environment  to  an 
unbounded  environment),  the  more  i mportant  developing  and  sharing  of  a  common 
vocabulary  will  be. 

4.3.3  Operations 

Thefinal  aspect  of  interoperability  relates  to  the  operations  that  communicating  enti¬ 
ties  might  perform  on  the  data  of  relevance— in  this  case,  data  related  to  a  schedule. 
Consideration  of  operations  leads  to  a  consideration  of  the  behaviors  of  the  commu¬ 
nicating  entities.  The  operations  act  on  some  state  data,  such  as  those  elements 
described  in  Section  4.3.2. 

What  are  the  behaviors  that  communicating  entities  might  perform  with  regard  to 
schedule?  Let  us  assume  for  the  moment  that  a  schedule  event,  such  as  a  milestone 
review,  has  a  designated  owner.  We  further  assume  the  owner  is  the  organization 
that  is  principally  responsible  for  the  event,  recognizing  that  other  organizations  may 
have  i  nterest  i  n  the  schedul  e  event. 

From  the  perspective  of  the  event  owner,  we  suggest  that  the  foil  owing  behaviors 
apply: 

•  assume  sole  responsibility  and  authority  for  making  changes  to  data  asso¬ 
ciated  with  a  schedule  event 

These  changes  could  include,  for  example,  creating  and  modifying  the  date  for 
some  schedule  event. 

•  notify  other  entities  of  any  change  to  a  schedule  event 

There  should  be  a  deadline  when  notifications  should  be  provided  to  entities  that 
require  such  schedule  information. 

•  maintain  a  history  of  changes  to  schedule  events 

Historical  data  can  be  useful  when  multiple  programs  are  involved.  For  example, 
analysis  of  such  data  could  be  used  to  indicate  the  presence  of  possible  problems. 
This  kind  of  analysis  is  analogous  to  the  use  of  past  performance  information; 


12.  In  fact,  it  is  also  possible  to  include  an  approval  date  itself  as  an  event  on  some  schedule.  In  this  case  there  is  a  depen¬ 
dency  between  the  schedule  that  contains  approval  information  and  the  schedule  that  includes  an  event  for  which  approv¬ 
al  is  being  sought.  There  are  also  variations  in  what  is  being  approved.  It  could  be  the  choice  of  a  particular  date,  or  the 
information  that  will  be  provided  on  that  date,  such  as  the  result  of  an  exit  criteria. 


26  |  CMU/SEI-2006-TN-035 


however,  for  an  ongoing  multi  project  acquisition,  we  would  introduce  a  new 
term— present  performance  information! 

•  mediate  negotiation  about  possible  changes  to  a  schedule  event 

Given  our  assumption  that  the  schedule  event  has  a  single  owner,  thefirst  two  behav¬ 
iors  are  obvious.  The  reason  to  maintain  a  history  of  changes  to  a  schedule  event  is 
that  analysis  of  those  changes  might  provide  insight  into  the  overall  management 
process.13  Finally,  we  have  assumed  that  the  owner  of  a  schedule  event  is  responsible 
for  managing  (mediating  negotiation  about)  changes  to  a  schedule  event  because  the 
owner  is  most  likely  the  entity  with  the  greatest  interest  in  the  event. 

4.4  SUMMARY 

A  simple  summary  of  the  concepts  associated  with  a  schedule  is  presented  as  a 
semantic  net  and  shown  in  Figure  8. 


milestone  milestone 


Figure  8:  Semantic  Net  for  Schedule  Information 


The  primary  focus  of  Figure  8  is  to  show  some  types  of  schedule  events,  their  charac¬ 
teristics,  and  the  behaviors  that  organizations  may  perform.  No  doubt  a  diagram  such 
as  this  can  be  developed  in  greater  detail. 

Consider  what  happens  as  we  turn  our  attention  to  a  system  of  systems.  How  does  the 
information  shown  in  Figure  8  change?  We  suggest  it  can  be  represented  as  shown  in 
Figure  9.  It  is  assumed  in  that  figure  that  two  PMOs  (PMO  and  PMO')  have  a  sched¬ 
ule  dependency,  with  one  of  them  providing  information  to  an  integration  activity. 


13.  The  expression  overall  management  process  does  not  imply  a  centrally  managed  process;  peer-to-peer  management 
process  applies  as  well. 
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There  are  a  number  of  points  we  would  make  about  Figure  9,  centering  on  the  PM  Os' 
dependency  on  some  schedule  events.  In  particular,  we  suggest  this  dependency 
i  mpl  i  es  a  need  to  share 

•  characteristics  (such  as  those  shown  in  Figure  8) 

For  example,  if  the  schedule  event  for  one  program  changes,  it  is  incumbent  on 
that  program  to  provide  an  update  to  the  other  program(s). 

•  behaviors  that  can  causea  change  toa  characteristic  of  a  scheduleevent  resulting 
i  n  a  change  to  the  schedul  e  event 

For  example,  if  a  schedule  event  is  approved  in  the  context  of  one  program,  this 
information  should  be  provided  to  the  other  program  (because  it  may  then  be 
viewed  as  a  constraint  to  that  program).  Of  course,  one  would  hope  that  the 
approval  processes  are  negotiated  in  view  of  the  fact  that  there  is  a  dependency 
relation. 


schedule 


PMO-^ 


dependency 


►  PMO’ 


owns 


schedule 


approve  create  change  delete 


approve  create  change  delete 


Figure  9:  Semantic  Net  for  Interactions  Regarding  Scheduling 


The  dependency  on  schedule  events  between  programs  implies  a  dependency  between 
the  programs.  That  connection  is  obvious,  but  realizing  that  the  dependencies  are 
negotiated  is  less  obvious!  Figure  9  reflects  the  behaviors  that  PM  Os  would  have  to 
change  when  considering  the  context  for  acquisition  in  a  system-of-systems  environ¬ 
ment.  One  example  of  a  such  a  dependency  is  when  there  is  a  change  to  an  event  by 
one  program  that  should  involve  resolution  of  any  dependencies  with  other  programs. 
This  type  of  behavior  occurs  in  a  system-of-systems  context  but  may  be  absent  in  a 
system-specific  acquisition. 
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5  Reflection 


Based  on  the  discussion  in  Section  4  about  schedule  from  the  perspective  of  interoper¬ 
able  acquisition,  we  can  take  a  larger  perspective  to  provide  some  observations  and 
identify  a  number  of  research  issues  appropriate  to  this  area. 

5.1  SOME  OBSERVATIONS 

Dealing  with  schedule  is  like  the  tip  of  the  proverbial  iceberg.  One  simple 
approach  is  to  treat  a  schedule  as  a  temporal  sequence  of  events,  recognizing  that 
there  can  be  different  types  of  events.  This  approach  was  taken  in  Figure  6  on  page 
20.  However,  another  equally  valid  approach  is  to  consider  a  schedule  as  a  set  of  tem¬ 
poral  activities  that  includes  goals,  resources,  constraints,  and  events.  This  approach 
encompasses  a  much  broader  scope. 

Overall,  it  is  not  enough  to  define  schedule;  rather,  it  is  essential  to  determine  what  is 
needed  to  achieve  interoperability  between  entities  that  participate  in  an  acquisi¬ 
tion— the  factors  that,  I  ike  the  submerged  90%  of  an  iceberg,  are  critical.  Put  another 
way,  a  schedule  is  a  window  into  a  program. 

The  information  to  be  shared  regarding  schedule  is  problematic.  One 

result  of  the  discussion  in  Section  4  is  the  view  of  schedule  (using  the  particular 
case  of  an  event)  in  terms  of  its  associated  information.  Our  abstract  example  of 
this  approach  is  shown  in  Figure  6  on  page  20.  I  n  Figure  10,  we  drill  into  the 
structure  of  data  for  a  schedule  event  to  describe  core  (elements)  and  supple¬ 
mentary  (elements  and  details)  data.  The  type  of  the  event— a  decision  point  or 
an  executive  review,  for  example— is  important,  of  course. 

•  The  core  data  consists  of  those  elements  deemed  most  relevant:  a  name,  the 
name's  meaning,  a  date  and  probability  distribution  function,  a  contact,  a  status, 
and  information  about  the  approval  of  the  schedule  event.  Those  elements  were 
discussed  in  Section  4.3.2  on  page 24. 

•  Depending  on  the  type  of  a  core  data  element,  there  may  be  various  supplemen¬ 
tary  data  items  required  by  statute  or  regulation.  For  instance,  there  is  a  require¬ 
ment  to  demonstrate  compl  i  ance  wi  th  the  C I  i  nger-Cohen  Act  for  M  i  I  estones  A  and 
B.  Milestones  B  and  C  also  have  a  requirement  to  provide  SARs.  We  detailed  the 
supplementary  information  required  for  milestone  decisions  and  reviews  in  Sec¬ 
tion  4.2  on  page  20. 

Should  such  information  be  made  avail  able  to  others?  From  the  perspective  of 
a  milestone  as  a  date  in  time,  some  might  argue  that  such  additional  data  is 
not  of  particular  relevance.  But  others  would  argue  that  achieving  interopera¬ 
ble  acquisition  is  about  sharing  information  and  allowing  others  to  gain  bene¬ 
fit  from  it.  Hence,  share  everything!  We  believe  that  supplementary  data 
belongs  in  the  basic  information  category,  because  it  relates  to  the  approval  of 
a  milestone.  Consequently,  it  assumes  greater  importance. 
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The  scope  and  extent  of  information  (core  and  supplementary)  regarding 
schedule  is  possibly  larger  than  one  might  like.  A  schedule  event  might  be 
thought  of  as  a  milestone  with  a  name  and  some  representation  for  its  temporal  char¬ 
acter,  at  first.  We  have  represented  the  temporal  character  as  a  probability  distribu¬ 
tion  function  rather  than  a  single  point  in  time.  We  then  expanded  the  boundary  of 
coreschedule  information  to  include  data  relating  to  the  approval  of  a  schedule  event. 
Some  of  the  approval  data  might  be  relevant  and  should  be  shared  (see  Section  4.3.2). 
F  urther,  there  may  be  supplementary  data  artifacts  for  some  schedule  event, 
required  by  statute  or  regulation  as  we  have  described. 


Event 


Figure  10:  Structure  of  Data  for  an  Event 


The  scope  of  the  data  could  even  extend  beyond  the  artifacts  required  by  regulatory 
processes.  Consider  the  case  of  risk  management.  It  is  quite  likely  that  some  entity 
would  inquire  about  the  risks  that  could  adversely  impact  some  schedule  event. 
Should  such  information  be  provided  by  the  owner  of  the  schedule  event?  Moreover, 
there  is  the  very  real  possibility  that  there  may  be  shared  risks  about  some  schedule 
event;  how  should  they  be  treated  in  this  context? 

One  can  go  even  further.  It  is  possible  that  a  program  should  share  information  about 
the  products  it  is  using,  or  has  selected,  such  as  vendor  (and  license)  information.  The 
reason  to  share  product  information  is  that  another  program  might  find  it  valuable. 
Thus,  sharing  of  this  information  could  prove  useful  to  another  program. 

The  question  of  authority  is  problematic.  Recall  the  discussion  concerning 
behaviors  regarding  schedule  events  in  Section  4.3.3.  There,  it  was  assumed  that  a 
single  entity  had  the  responsibility  and  sole  authority  to  make  changes  to  a  schedule 
event.  Is  this  the  right  model,  or  does  it  go  against  the  grain  of  network-centric  opera¬ 
tions  regarding  collective  behavior?  I  n  this  area,  we  are  closing  in  on  the  problem  of 
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centralized  versus  distributed  control.  There  must  be  a  clear  understanding  of  the 
concepts  of  ownership  and  authority,  as  well  as  their  areas  of  control.  In  fact,  one 
might  choose  to  also  consider  a  hierarchy  of  authority.14 

Dealing  with  the  unbounded  case  raises  new  questions.  The  subject  of  an 
unbounded  environment  was  discussed  in  Section  2.2;  in  particular,  seethe  discus¬ 
sion  regarding  Figure  2  on  page  6.  In  the  unbounded  case,  theremay  be  entities,  data, 
and  behaviors  that  are  only  known  at  runtime,  characteristic  of  a  dynamic  environ¬ 
ment.  It  is  not  possible,  a  priori,  to  negotiate  agreements  concerning  communication 
among  entities,  the  data  they  share,  and  the  behaviors  they  are  expected  to  perform. 
How  are  dependencies  managed  in  an  unbounded  environment,  recognizing  that  they 
might  have  implications  for  other  agreements? 

5.2  RESEARCH  QUESTIONS 

There  are  a  number  of  research  questions  that  arise  from  addressing  schedule  in  the 
context  of  interoperable  acquisition,  in  addition  to  those  mentioned  in  the  preceding 
section.  Schedule  is  but  one  aspect  of  interoperable  acquisition;  a  related  topic  is 
interoperable  risk  management.  A  number  of  research  questions  for  interoperable 
risk  management  arediscussed  in  Risk  Management  Considerations  for  Interoperable 
Acquisition  [M  eyers  06b],  The  questions  we  would  raise  here  are  similar  to  those. 

How  do  considerations  for  schedule  in  the  context  of  a  system-centric  acquisition  differ 
from  such  considerations  in  the  context  of  interoperable  acquisition? 

One  approach  to  identifying  the  role  of  a  schedule  is  the  use  of  a  checklist,  and  it  has 
been  previously  addressed  [Park  95],  Park's  work  addressed  topics  such  as  objectives 
of  estimates,  estimation  factors,  integrity  of  the  estimating  process,  historical  evi¬ 
dence,  and  changes.  These  same  topics  apply  to  the  context  of  interoperable  acquisi¬ 
tion. 

Given  that  general  similarity,  what  are  the  differences  when  one  moves  from  a  sys¬ 
tem-centric  to  an  interoperable-acquisition  context?lt  isan  oversimplification  to  say 
that  a  system-centric  perspective  just  needs  to  consider  the  larger  context  of  interop¬ 
erable  acquisition.  To  answer  the  question  adequately,  we  need  to  return  to  the 
approach  to  interoperability  and  address  the  communicating  entities,  information 
shared,  and  behaviors  performed.  All  of  these  are  different  for  interoperable  acquisi¬ 
tion  when  compared  to  system-centric  acquisition.  I  n  the  area  of  behaviors  alone, 
there  is  a  greater  need  to  share  information,  negotiate  agreements,  and  communicate 
status  among  entities  engaged  in  the  broader  context.  Many  of  the  foil  owing  ques¬ 
tions  probe  the  contextual  differences. 


14.  However,  we  remind  the  reader  of  the  tension  between  a  rigid  hierarchal  structure  and  the  structure  espoused  for  network¬ 
centric  operations.  In  the  latter  case,  decisions  are  made  at  lower  levels  (or  at  the  edge,  if  you  like),  subject  to  intent  pro¬ 
vided  by  some  higher  level  authority. 
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What  are  the  entities  that  need  to  participate  in  a  discussion  of  schedule  in  an  interop¬ 
erable  acquisition? 

We  identified  a  number  of  possible  entities,  such  as  a  PM  O  and  an  M  DA.  But  as  a  the 
scope  of  interest  in  a  schedule  enlarges,  what  other  organizations  are  affected  and 
what  are  their  roles  in  discussions  regarding  schedule?  I  n  an  unbounded  environ¬ 
ment,  all  the  entities  that  have  an  interest  in  schedule  information  might  not  be 
known. 

What  information  regarding  schedule  needs  to  be  shared? 

The  topic  of  schedule  information  was  discussed  in  Section  4.3.2.  There,  it  was  sug¬ 
gested  that,  as  a  minimum,  thefollowing  items  warrant  consideration  for  sharing 
about  a  schedule  event  of  a  given  type: 

•  name 

•  meaning 

•  contact 

•  date  and  pdf 

•  status 

•  approval  information 

Yet,  despite  the  apparent  simplicity  of  that  suggestion,  there  are  questions  about 
these  items.  Consider  the  information  regarding  the  approval  of  a  schedule  event. 
Some  questions  that  come  to  mind  are  these: 

•  What  is  the  appropriate  state  model  for  the  approval  status?  Are  the  values 

U napproved,  in_Progress,  and  Approved  sufficient?  Or,  should  a  more  elaborate 
state  model  be  employed  that  might  include  states  for  Complete  or  Pending ? 
These  questions  are  particularly  relevant  for  interoperability:  If  a  different 
project  uses  different  values,  a  conflict  in  interpretation  is  possible.15 

•  Should  the  possibility  that  the  individual  having  decision  authority  may  delegate 
this  authority  to  someone  else  be  included? 

•  Is  the  concept  of  joint  approval  relevant? 

•  What  is  the  proper  choice  for  a  contact?  Should  it  be  a  set  of  individuals?  Should 
there  be  contacts  for  different  aspects  of  a  schedule?  Each  event  could  have  a  dif¬ 
ferent  contact. 

•  Should  approval  information  be  considered  part  of  the  core  information  about  a 
schedule? 

Another  way  to  approach  these  issues  is  to  apply  the  concept  of  an  ontology  for  a 
schedule  in  a  multi  program  context. 

What  is  the  distinction,  if  any,  between  owner  and  authority? 

Most  would  assume  that  an  owner  of  some  schedule  event  has  authority  over  the 
event  and  may  therefore  make  changes.  H  owever,  this  is  not  true  even  for  the  case  of 


15.  The  status  of  a  schedule  event  is  not  defined  in  any  specifications  we  are  aware  of. 
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a  particular  program  (think  of  the  role  of  an  M  DA  in  approving  milestone  decisions 
for  a  program).  I  n  the  context  of  a  traditional  system-centric  acquisition  the  relation 
between  owner  and  authority  is  specified.  However,  in  the  case  of  interoperable 
acquisition,  these  lines  of  demarcation  may  not  be  so  clear.  Ownership  and  authority 
may  need  to  be  viewed  as  a  shared  responsibility. 

What  are  the  processes  for  schedule  management  in  interoperable  acquisition? 

The  role  of  process  in  regard  to  various  aspects  of  software  engineering  is  well 
known.16  But  what  arethe  process  implications  with  regard  to  schedule  manage¬ 
ment?  For  example,  if  an  acquisition  organization  uses  a  process  for  schedule  man¬ 
agement,  must  other  organizations  involved  in  the  interoperable  acquisition  use  that 
same  process?  Or,  instead,  must  there  be  a  way  for  the  inputs  and  outputs  of  those 
processes  to  be  understood? 

What  are  the  behaviors  necessary  to  be  performed  by  entities  regarding  schedule  man¬ 
agement  in  an  interoperable  acquisition  context? 

This  interesting  question  goes  to  the  heart  of  achieving  interoperable  acquisition. 
What  set  of  requirements  would  describe  the  expected  behavior  of  entities  collaborat¬ 
ing  with  regard  to  schedule  information?  For  example,  consider  the  candidate 
requirements  shown  in  Figure  11. 


3.4.1 :  Exchange  of  Schedule  Information 

An  organization  participating  in  exchange  of  information  regarding  schedule  shall  conform  to 
the  following  requirements: 

1.  It  shall  permit  exchange  of  information  regarding  milestones  that  includes  the  name  of  the  mile¬ 
stone,  semantics,  date  and  probability  distribution  function,  and  a  point  of  contact. 

2.  Organizations  shall  be  capable  of  exchanging  information  regarding  the  approval  status  of  a 
specified  milestone. 

3.  There  shall  be  a  means  to  determine  the  confidence  in  elements  of  a  schedule.  The  confidence 
information  shall  be  made  available  to  others. 

4.  Upon  receipt  of  a  request  for  information,  the  organization  responsible  for  managing  a  schedule 
item  (e.g.,  milestone)  shall  provide  such  information  within  one  day. 

5.  The  requestor,  nature  of  the  requested  information,  and  the  time  interval  between  request  and 
response  for  schedule  information  shall  be  recorded. 


Figure  11:  Sample  Requirements  for  Exchange  of  Schedule  Information 


At  first  sight,  these  requirements  appear  to  be  innocuous,  but  they  illustrate  some 
points  raised  earlier: 

•  The  sample  requirements  presume  the  semantics  of  terms  such  as  schedule  item 
and  probability  distribution  are  understood. 


1 6.  See  for  instance  CMMI:  Guidelines  for  Process  Integration  and  Product  Improvement  [Chrissis  03]  and  Adapting  CMMI 
for  Acquisition  Organizations:  A  Preliminary  Report  [ Dodson  06]. 
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•  The  syntactic  representation  of  the  basic  schedule  items  must  be  shared.  Are  they 
represented  as  a  sequence  of  characters  (e.g.,  ASCI  I  or  Unicode)  or  in  another 
way? 

•  There  is  a  requirement  for  a  one-day  response.  Suppose  this  were  changed  to  a 
response  in  five  minutes?  If  so,  we  are  moving  into  a  domain  of  autonomous 
agents  that  parti  ci  pate  i  n  the  process. 

•  H  ow  mi  ght  the  i  nformati  on  mai  ntai  ned  about  the  request  and  response  be  used  i  n 
a  larger  context?  This  question  alludes  to  the  use  of  metrics  to  assess  the  collec¬ 
tive  process. 

•  What  about  the  trustworthiness  of  the  data  provided? 

What  is  the  role  of  an  integrated  master  schedule  in  a  system-of-systems  acquisition 
environment? 

An  integrated  master  schedule  (I  MS)  is  a  well-known  concept  and  is  frequently  called 
out  on  a  contract.  An  I  MS  shows  tasks  and  their  timing;  it  typically  contains  contract 
milestones,  accomplishments,  tasks,  and,  thus,  the  relation  among  activities  and 
events  in  some  organization.  The  data  item  description17  for  an  I  MS  defines  it  as  "an 
i  ntegrated  schedul e  contai  ni ng  the  networks,  detai I ed  tasks  necessary  to  ensure  suc¬ 
cessful  program  execution"  [DoD  05b],  The  information  about  I  MS  is  typically  in  the 
context  of  a  particular  program.  How  do  things  change  when  one  considers  interoper¬ 
able  acquisition?  Can  the  concept  of  a  system-centric  I M  S  be  extended  to  the  larger 
context? 

How  does  the  concept  of  schedule  viability  apply  in  the  context  of  interoperable  acqui¬ 
sition? 

Loosely  speaking,  a  schedule  is  viable  if  the  dates  and  their  associated  requirements 
can  be  achieved.  How  does  this  concept  extend  to  the  scope  of  interoperable  acquisi¬ 
tion  where  the  number  of  participating  organizations  may  become  quite  large?  I  ndi- 
vidual  schedules  might  be  viable  but  their  composition  not  viable,  because 
dependencies  between  the  schedules  must  be  accounted  for.  Who  is  responsible  for 
determining  the  viability  of  that  larger  schedule? 

What  are  the  metrics  to  assess  interoperability  with  regard  to  schedule? 

The  goal  of  an  organization  that  professes  to  be  a  high-maturity  organization  is  to 
perform  in  an  optimal,  quantitatively  managed  manner.  When  one  considers  a  sys¬ 
tem  of  systems,  the  question  of  optimality  becomes  problematic:  optimal  for  a  particu¬ 
lar  organization  may  besuboptimal  for  the  system  of  systems.  However,  the  question 
of  quantitative  measures  to  assess  the  overall  management  is  still  relevant.  One 
approach  to  this  question  is  through  the  use  of  metrics. 


17.  In  general,  a  DoD  data  item  description  (DID)  spells  out  the  deliverable  data  required  of  a  contractor. 


34  |  CMU/SEI-2006-TN-035 


It  is  interesting  to  examine  this  question  from  two  perspectives.  First,  consider  the 
entity  responsible  for  managing  a  schedule  in  some  acquisition  context.  Some  possible 
metrics  in  this  case  might  include 

•  creation  of  any  new  schedule  events 

•  changes  toan  attribute  associated  with  any  existing  schedule  event,  includingthe 
date  of  the  schedule  event 

Now,  consider  the  perspective  that  addresses  those  other  entities  with  interest  in  or 
knowledge  of  some  schedule  event.  Here  metrics  could  measure  the 

•  number  of  schedule  events  that  occur  within  bounds  defined  by  the  associated  pdf 

•  number  of  requests  for  information  about  basic  schedule  attributes 

•  interval  of  time  between  the  request  for  schedule  information  and  its  response 

•  number  of  requests  to  change  some  schedule  attribute 

•  number  of  requests  for  changes  in  some  schedule  attribute  that  were  granted. 
This  latter  case  raises  the  question  about  the  ripple  effects  that  can  impact  other 
parti  ci  pants. 

Potentially,  a  wealth  of  information  can  be  captured  regarding  schedule  information. 
For  example,  consider  the  hypothetical  data  shown  in  Figure  12. 
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Figure  12:  Hypothetical  Schedule  Metrics 


I  n  part  (a)  of  the  figure,  we  show  that  the  number  of  requests  made  for  information 
regarding  schedule  is  relatively  large  and  constant.  The  high  and  consistent  volume 
might  indicate  the  desire  of  communicating  entities  to  be  aware  of  schedule  informa- 
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tion.  Part  (b)  shows  the  time  interval  between  schedule  information  requests  and  the 
response.  This  largely  constant  function  indicates  that  the  process  of  responding  to 
requests  is  well  managed.  I  n  part  (c),  the  number  of  requests  to  change  schedule 
information  is  shown;  this  function,  although  it  fluctuates,  remains  relatively  high. 
Finally,  in  part  (d),  the  number  of  changes  to  schedule  information  is  shown.  Notice 
that  this  function  decreases  with  time— indicative,  perhaps,  of  a  need  to  stabilize  the 
(collective)  process  by  not  granting  changes  to  a  schedule. 

Although  examples  of  information,  such  as  those  in  Figure  12,  are  interesting,  the  col¬ 
lective  perspective  is  of  greater  interest.  In  regard  to  the  hypothetical  data,  one  might 
conclude  that  'The  interaction  between  entities  for  schedule  information  is  relatively 
high,  the  responses  are  good,  but  there  were  a  lot  of  requests  for  changes  that  were 
denied.  This  analysis  indicates  the  individual  entities  were  seeking  (too  much)  sched¬ 
ule  relief  and  that  there  are  problems  at  the  local  management  level." 

That  conclusion  illustrates  one  inference  about  the  sharing  of  information  regarding 
schedule  among  organizations— an  inference  based  on  the  data  shared  and  reflecting 
the  behaviors  of  communicating  entities  engaged  in  acquisition  in  a  system-of-sys- 
tems  environment.  I  n  that  sense,  the  appropriate  metrics  can  be  quite  valuable  in 
helping  to  gain  insight  into  the  overall  acquisition.  We  say  "appropriate"  metrics 
because  the  wrong  metrics  could  contribute  to  unintended  interpretations  of  perfor¬ 
mance. 

We  find  discussions  of  metrics  and  their  use  to  be  of  special  importance  to  a  system  of 
systems.  The  discussion  of  metrics  regarding  schedule  might  form  part  of  a  more  gen¬ 
eral  question  about  metrics  for  the  collaboration  of  organizations.  For  example,  the 
hypothetical  metrics  shown  for  schedule  in  Figure  9  on  page  28  could  also  be  pre¬ 
sented  for  topics  such  as  cost  considerations  or  risk  sharing.  I  n  each  case,  the  metrics 
would  reflect  the  collective  behaviors  regarding  these  topics.  Our  discussion  of  met¬ 
rics  goes  beyond  the  traditional  scope  (i.e.,  in  the  context  of  a  particular  organization) 
of  metrics  to  emphasize  a  sharing  of  information  and  management  opportunities. 
Thus,  the  use  of  metrics  is  but  another  aspect  of  the  different  characteristic  of  a  sys- 
tem-of-systems  environment. 

How  can  the  sharing  of  history  data  be  used  to  advantage  when  multiple  organizations 
are  engaged  in  an  acquisition? 

This  question  arose  in  the  discussion  of  behaviors  in  Section  4.3.3.  There,  we  intro¬ 
duced  the  concept  of  present  performance  information  to  allude  to  a  possible 
advantage  of  sharing  information— namely,  that  such  sharing  would  benefit  the  enti¬ 
ties  that  collaborate  in  an  acquisition.  Note  also  the  close  relation  of  sharing  history 
data  to  the  role  of  metrics.  If  it  were  possibleto  maintain  and  share  history  informa¬ 
tion  about  acquisition  programs  engaged  in  interoperable  acquisition,  it  might  be  pos¬ 
sibleto  make  corrections  to  an  acquisition(s)  during  the  course  of  a  program 
execution. 
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How  much  automated  support  can  be  provided  for  schedule  considerations  in  the  con¬ 
text  of  interoperable  acquisition? 

Weview  this  question  as  extremely  important.  Tothe extent  possible,  thereis  no 
need  for  a  person  to  intervene  in  managing  schedule  information  in  an  interoperable 
acquisition  context.  For  example,  if  some  entity  is  interested  in  schedule  information, 
this  should  be  provided  without  involvement  by  a  person. 

H  owever,  there  are  questions  about  the  utility  of  autonomous  agents. 

•  Do  they  have  enough  knowledge  to  identify  and  carry  out  a  particular  task? 

•  Do  they  have  access  (and  ri  ghts)  to  rel  evant  i  nformati  on? 

•  Would  a  decision-making  capability  be  embedded  in  an  agent? 

•  How  is  an  agent  notified  of  changes  tothe  environment? 

•  How  can  agents  be  kept  current  in  a  dynamically  changing  environment? 

•  What  happens  in  an  unbounded  environment? 

We  also  would  enlargethe  discussion  of  where  autonomous  agents  can  play  an  active 
role  in  managing  schedule  information.  For  example,  consider  an  autonomous  process 
that  is  invoked  whenever  a  change  to  some  schedule  event  occurs.  This  process 
assesses  the  potential  impact  of  the  change  on  some  program  and  may  take  actions. 
Perhaps  for  a  small  change,  no  action  is  likely  required.  (Of  course,  many  small 
changes  can  have  an  adverse  cumulative  effect.)  However,  if  the  change  affects  the 
entity  in  a  serious  way,  the  action  performed  by  the  autonomous  agent  is  commensu¬ 
rate  with  the  nature  of  the  change.  As  systems  become  more  i  ntertwi  ned  and  need  to 
support  greater  interoperability,  the  process  of  managing  and  using  schedule  infor¬ 
mation  should  be  made  as  automated  as  possible.  There  are  also  concerns  about  areas 
such  as  bandwidth  and  security. 

However,  increased  information  exchange  by  machine  (perhaps  even  in  the  case  of 
autonomous  agents)  raises  new  issues.  I  n  particular,  there  needs  to  be  agreement  on 
a  communication  protocol.  The  protocol  must  address,  at  a  minimum,  the  syntax  and 
semantics  of  the  information  exchanged.  We  established  the  importance  of  semantics 
by  suggesting  that  it  is  one  of  the  basic  attributes  of  a  schedule  event  (see  Section 
4.3.2).  However,  we  did  not  view,  in  that  context,  the  syntax  of  that  information  to  be 
of  equal  importance.  But  when  machine  communication  is  applied,  the  syntax  of 
schedule  information  must  be  addressed. 

The  use  of  an  autonomous  agent  can  be  viewed  as  but  one  example  of  a  decision  aid 
that  can  assist  in  schedule  management  in  an  interoperable  acquisition  context. 
There  are  other  decision  aids  that  can  also  be  applied.  For  example,  a  decision  aid 
that  can  assist  in  tracking  and  maintaining  the  state  of  multiple  acquisitions  would 
be  useful  (as  opposed  to  an  individual  making  changes  to  a  large  wall -si zed  Gantt 
chart). 
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What  are  the  practical  implications  of  this  work? 

One  aspect  of  interoperability  is  the  sharing  of  information  about  items  of  interest.  I  n 
that  sense,  it  is  natural  to  speculate  about  how  various  types  of  organizations  would 
respond  to  requests  for  information  sharing— or,  more  to  the  point,  about  the  degree 
of  information  they  would  share.  On  the  one  hand,  communicating  entities  need  to 
develop  a  common  understanding  of  certain  key  concepts,  such  as  information  about 
schedule.  On  the  other  hand,  an  organization  might  not  enter  into  an  agreement 
about  sharing  schedule  information,  to  avoid  losing  a  competitive  advantage.  For 
example,  some  organization  may  believe  it  has  a  robust  and  accurate  cost  estimating 
process  that  gives  it  a  competitive  advantage.  It  is  one  thing  to  share  the  results  of 
that  process  (such  as  the  variance  of  a  schedule  event)  but  quite  another  to  expose  the 
details  of  the  process. 

One  approach  to  this  question  would  be  to  survey  organizations  of  various  types  to  see 
what  information  they  would  share  and  why  they  would  share  it.  No  doubt,  an  organi¬ 
zation's  answers  would  depend  on  its  type(e.g.,  PMO  or  contractor).  Perhaps  the 
extent  to  which  organizations  are  willing  to  compromise  in  the  best  interest  of  a  sys¬ 
tem  of  systems  wi  1 1  i  nfl  uence  thei  r  answers,  too. 

Questions  related  to  schedule  concerns  (such  as  those  listed  in  Section  3.2)  can  be 
useful.  Those  questions  can  be  used  for  assessment,  as  indicated  earlier.  They  repre¬ 
sent  a  starting  point  for  eliciting  material  about  multiple  organizations  that  must 
interoperate.  In  fact,  questions  in  this  section  also  apply  in  that  context.  A  caution  is 
needed  here,  though:  There  may  be  parameters  (e.g.,  should  the  type  of  contract,  or 
the  domain  of  application,  be  considered  as  a  parameter?)  that  can  influence  many 
possible  results. 

How  can  cultural  concerns  be  accounted  for? 

This  question  is  closely  related  to  the  practical  implications  of  this  work.  For  exam¬ 
ple,  it  is  necessary  to  consider  what  behaviors  should  be  rewarded  and  what  possible 
changes  to  the  reward  system  would  foster  success  in  interoperable  acquisition.  One 
would  expect  that  the  culture  of  acquisition  in  a  system-of-systems  environment  must 
encourage  the  sharing  of  information  as  well  as  the  collective  behaviors  that  are  ori¬ 
ented  toward  goals  reflecting  a  system  of  systems,  rather  than  one  particular  system. 

To  what  degree  can  the  information  about  schedule  be  applied  to  other  areas,  such  as 
cost? 

I  n  Section  3,  we  listed  a  number  of  questions  that  shed  light  on  various  aspects  of  a 
schedul  e.  T wo  questi  ons  were  these:  "What  are  the  ri  sks  to  your  meeti  ng  the  schedul  e 
event?  Are  the  risks  shared,  and  managed,  among  all  PM  s  and  the  integrator?"  What 
if  schedule a/ent  were  replaced  with  the  word  cost?  We  suggest  that  the  questions 
posed  in  Section  3  can  be  adapted  to  other  items  of  interest  to  acquisition  in  a  system- 
of-systems  context.  The  portability  of  thethose  questions  means  that  they  can  serve 
as  a  model  that  could  be  applied  toother  domains. 
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Does  the  concept  of  a  dependency  framework  apply  to  this  and  other  work  regarding  a 
system  of  systems? 

We  introduce  the  concept  of  a  dependency  framework  as  a  contextual  means  to 
understand  dependencies  (and  influence  relations).  Such  dependencies  need  to  be 
defined  and  their  implications  understood.  Defining  and  understanding  them 
involves  consideration  of  data  and  behaviors,  among  other  things.  I  n  the  simplest 
case,  an  organization  may  have  a  dependency  on  some  other  organization,  reflected  in 
a  schedule. 

We  believe  that  much  can  be  done  in  dependency  management  and  that  it  is  of  special 
relevance  to  interoperable  acquisition.  For  example,  consider  the  notation 
Depends_On  (A,  B)  to  denote  that  A  depends  on  B.  If  there  is  another  dependency 
DependsjOn  (B,  C),  one  may  be  tempted  to  assume  that  it  follows  that  DependsjOn 
(A,  C)  is  also  true.  Understanding,  expressing,  and  managing  dependency  relations  is 
expected  to  be  important  where  one  is  concerned  with  interoperable  acquisition. 
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6  Summary 


This  report  examined  the  role  of  schedule  in  the  context  of  interoperable  acquisition. 
The  importance  of  scheduletothe  acquisition  of  an  individual  system  is  multiplied— 
considerably— as  one  considers  a  system-of-systems  environment.  This  report  has 
taken  a  step  in  the  direction  of  examining  how  to  deal  with  schedule  in  interoperable 
acquisition. 

We  believe  there  are  several  interesting  aspects  of  this  work  with  regard  to  schedule 
in  a  system-of-systems  acquisition  environment,  including  the  foil  owing: 

•  A  change  from  a  system-centric  environment  to  a  system-of-systems  environment 
motivates  the  need  to  share  data  and  engage  in  collective  behavior.  This  point  was 
illustrated  in  our  discussion  of  Figure9on  page  28  that  shows  dependencies  must 
be  understood  and  managed. 

•  The  connection  between  sharing  of  schedule  information  and  risk  management 
could  be  exploited  to  advantage.  For  example,  knowledge  of  possible  risks  and 
their  implication  on  schedule  is  desirable.  This  coupling  of  risks  and  their  sched¬ 
ule  implications  is  necessary  to  gain  a  more  realistic  picture  of  acquisition. 

•  The  question  of  metrics  is  very  interesting.  The  discussion  concerning  Figure  12 
on  page  35  illustrates  possible  metrics  for  schedule  in  the  broader  context.  Of 
greater  interest  are  the  inferences  of  collective  behaviors  that  may  shed  light  on 
the  progress  of  an  interoperable  acquisition. 

There  are  several  opportunities  for  follow-on  work  identified  here.  Among  them,  we 
would  note  the  foil  owing: 

•  How  can  the  research  questions  posed  in  Section  5  regarding  communicating  enti¬ 
ties,  information  shared,  processes,  the  role  of  automating  schedule  management, 
and  other  topics  be  pursued  to  meet  the  goal  of  more  effective  acquisitions  in  a 
system-of-systems  context? 

•  To  what  extent  can  this  work,  focused  on  schedule,  be  applied  toother  areas?  How 
much  of  the  approach  used  here  can  be  reused  in  another  context?  What  about  the 
concept  of  cost,  for  instance?  What  is  the  interaction  between  cost  and  schedule, 
and  how  should  that  connection  be  addressed? 

•  Can  an  assessment  process  be  developed  that  addresses  schedule  events  when 
there  are  multiple  organizations  involved?The  example  of  a  Gedanken  red  team, 
presented  in  Section  3,  is  a  start  in  this  direction.  However,  a  more  general 
approach  is  warranted,  so  that  it  can  be  applied  to  other  areas. 

It  is  worth  noting  that  we  have  considered  schedule  in  the  context  of  acquisition,  but 
the  importance  of  a  schedule  extends  to  other  areas,  significantly  in  the  construction 
and  operation  of  systems.  This  recognition  highlights  the  need  to  understand  and 
exchange  information  about  schedule  across  those  areas  (which  we  termed  aspects  of 
interoperability  in  Section  2.2)  to  achieve  interoperability.  Hence,  we  believe  that 
much  of  this  work  can  be  applied  to  areas  other  than  acquisition. 
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We  anticipate  the  work  outlined  here,  as  well  as  further  work  related  to  the  concept  of 
schedule  management  in  a  system-of-systems  environment,  will  be  pursued. 
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Appendix  A  Acronyms  and  Initialisms 


AoA 

Analysis  of  Alternatives 

C4I 

Command,  Control,  Communications,  Computers,  and  I  ntelligence 

CD 

Concept  Decision 

CR 

Concept  Review 

CDD 

Capability  Development  Document 

CMMI 

Capability  Maturity  Model  Integration 

COTS 

Commercial  off-the-shelf 

DAB 

Defense  Acquisition  Board 

DAU 

Defense  Acquisition  University 

DID 

Data  Item  Description 

DoD 

Department  of  Defense 

DRR 

Design  Readiness  Review 

ETVX 

Entry,  Task,  Validation,  Exit 

FAR 

Federal  Acquisition  Regulation 

FRP 

F  ull  Rate  Production 


ICD 

Initial  Capabilities  Document 
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IMS 

I  ntegrated  M  aster  Schedule 

IR&D 

I  ndependent  Research  and  Development 

ITAB 

I  nformation  T echnology  Acquisition  Board 

JROC 

J  oint  Requirements  Oversight  Council 

LFT&E 

Live  Fire  Testing  and  Evaluation 

LRIP 

Low  Rate  I  nitial  Production 


MDA 

Milestone  Decision  Authority 

MOU 

M  emorandum  of  U  nderstandi ng 

pdf 

probability  distribution  function 

PEO 

Program  Executive  Office 

PPBES 

Planning,  Programming,  Budgeting,  and  Execution  System 

PM 

Program  Manager 

PMO 

Program  Management  Office 

Pub.  L. 

Public  Law 


SEI 

Software  E  ngi  neeri  ng  I  nstitute 

SAR 

Selected  Acquisition  Report 

SOSI 

System-of-Systems  I  nter operability 
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TD 

T echnology  Development 

use 

United  States  Code 
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